The present invention relates to a multiplexer, multiplexing method and a recording medium for a video coded stream.
Generally, a recording and reproducing apparatus such as an optical disc drive comprises an encoder and decoder for compression and expansion (or decompression) based on the Moving Pictures Experts Group (the xe2x80x9cMPEGxe2x80x9d) standard. The apparatus records and reproduces a video signal through the MPEG-based image compression or expansion.
In the MPEG method, pictures (frames or fields) composing together a moving image are coded into any of types: I-picture, P-picture and B-picture in order to compress the image.
The I picture is an xe2x80x9cintra-coded picturexe2x80x9d. It is produced by spacial coding within a picture without reference to other pictures. Therefore, the I picture is used as a random-access entry point or for recovery from an error.
The P picture is a xe2x80x9cpredictive-coded picturexe2x80x9d. It is produced by coding through prediction from a previous I or P picture. Therefore, to decode a P picture, the past I or P picture must be decoded.
The B picture is a xe2x80x9cbidirectionally predictive-coded picturexe2x80x9d. It is produced by coding through forward, backward or bidirectional prediction from a previous I or P picture and a later I or P picture. Therefore, to decode a B picture, the previous and later I or P pictures must be decoded.
Thus, the MPEG method enables efficient compression of a moving image by inter-picture predictive coding and random access to a compressed moving image.
Also, the MPEG method compresses a moving image into a data stream formed from data in units of Group of Pictures (xe2x80x9cGOPxe2x80x9d), each GOP containing an arbitrary number of I, P and B pictures. It is prescribed in the MPEG method that a GOP should include at least an I picture. Therefore, random access can be made to a moving image compressed in units of GOP.
Here, it will be discussed how the aforementioned recording and reproducing apparatus reproduces an image signal compressed by the MPEG method.
It is assumed, for example, that the recording and reproducing apparatus uses a recording medium having recorded thereon a stream of coded data as shown in FIG. 10A. The recording and reproducing apparatus decodes the data stream recorded as shown in FIG. 10A and displays pictures in the data stream in an order shown in FIG. 10B. The symbols I, P and B added to the pictures indicate an I picture, P picture and B picture, respectively. Each of the symbols has a suffix which is a so-called temporal reference indicative of the order of each picture in the GOP.
To reproduce coded data in the data stream shown in FIG. 10A, first, a picture I0 is decoded. Since an I picture is produced by spacial coding within a picture without reference to other pictures, as mentioned above, the picture I0 can be decoded individually without decoding other pictures. Next, a picture P2 produced by forward predictive coding is decoded based on decoded picture I0. Since a P picture is produced by coding through prediction from a previous I or P picture, picture I0 must be decoded before picture P2 is decoded. Then, a picture B1 produced by bidirectional prediction coding is decoded based on decoded pictures I0 and P2. Since a B picture is produced by coding through bidirectional prediction from a previous I or P picture and a later I or P picture, pictures I0 and P2 must be decoded before picture B1 is decoded. In this way, the coded data in the data stream shown in FIG. 10A is decoded in the order of I0 xe2x86x92P2xe2x86x92B1xe2x86x92P4xe2x86x92B3xe2x86x92P6xe2x86x92B5xe2x86x92I8xe2x86x92B7xe2x86x92P10xe2x86x92B9xe2x86x92 . . . .
To display each of these decoded pictures, their decoded order is changed to I0xe2x86x92B1xe2x86x92P2xe2x86x92B3xe2x86x92P4xe2x86x92B5xe2x86x92P6xe2x86x92B7xe2x86x92I8xe2x86x92B9xe2x86x92P10xe2x86x92 . . . as shown in FIG. 10B.
If the recording medium is random-accessible, coded data recorded by the MPEG method can be random-accessed. Therefore, previously, pictures down to a P picture at point SA in a stream shown in FIG. 11 for example may be reproduced, subsequent pictures may be left not reproduced down to a picture just before a B picture at point SB (picture B3), and the picture B3 and subsequent pictures may be reproduced. That is, a stream can be reproduced with some of the pictures in the stream left not reproduced. Leaving some pictures not reproduced will be referred to as xe2x80x9cskippingxe2x80x9d hereinafter, and reproducing a stream while skipping some pictures will be referred to as xe2x80x9cskipping reproductionxe2x80x9d hereinafter. Also, a picture just before the skipping is started (the picture at point SA in FIG. 11, for example) will be referred to as an xe2x80x9cout-point picturexe2x80x9d hereinafter, while a picture which is first reproduced after completion of the skipping (the picture at point SB shown in FIG. 11, for example) will be referred to as an xe2x80x9cin-point picturexe2x80x9d hereinafter.
In skipped reproduction, an image thus reproduced may be temporally discontinuous in some cases.
If an in-point picture is a B picture for example, an I or P picture necessary for decoding this B picture must be decoded. In this case, the reproduced B picture image may be temporally discontinuous if the necessary I or P picture is not decoded. This problem will be described in view of the example shown in FIG. 11. Since the in-point picture is a picture B3, at least pictures I0, P2 and P4 must be decoded to decode picture B3. Therefore, while pictures I0, P2 and P4 are being decoded, no pictures can be displayed. Thus, an image resulting from such a skipping reproduction would be temporally discontinuous. Consequently, if a skipping reproduction is effected, pictures before and after skipped pictures cannot be reproduced so as to be seamlessly contiguous to each other.
In the foregoing, image compression by the MPEG method has been described. In addition to the aforementioned methods of predictive coding, there is a method of inter-picture predictive coding based on the fact that two successive pictures in a stream have a correlation between them. In the inter-picture predictive coding, a difference between the two pictures is determined and coded. Even by this method, an image resulting from a skipping reproduction may be temporally discontinuous.
Accordingly, it is an object of the present invention to overcome the above-mentioned drawbacks by providing a multiplexing technique for coding a moving image with skip over pictures in the image from one skipping point to the next, so that the coded image can be reproduced with no temporal discontinuity between the skipping points.
It is another object of the invention to provide a multiplexing technique for coding a moving image such that a video buffer of a moving image decoder does not overflow or underflow during reproduction of the coded image.
The above objects can be attained by encoding a first video coded stream of which the display ends with a first picture and a second video coded stream of which the display begins with a second picture which is displayed following the first picture during skipping reproduction. The first video coded stream and an audio coded stream synchronous with the first video coded stream are packed to generate a first multiplexed stream, and the second video coded stream and an audio coded stream synchronous with the second video coded stream are packed to generate a second multiplexed stream. The first and second video coded streams are encoded in such a manner that for a period from the time at which the second video coded stream starts being supplied to a video buffer of a decoder until the time at which the decoding of the last coded picture in the first video coded stream ends, the bit occupancy in the video buffer is smaller than the capacity of the video buffer and larger than zero.